1. Field of the Invention
The present invention relates generally to information processing environments and, more particularly, to improved methods for self-tuned parallel database recovery.
2. Description of the Background Art
Computers are very powerful tools for storing and providing access to vast amounts of information. Computer databases are a common mechanism for storing information on computer systems while providing easy access to users. A typical database is an organized collection of related information stored as “records” having “fields” of information. As an example, a database of employees may have a record for each employee where each record contains fields designating specifics about the employee, such as name, home address, salary, and the like.
Between the actual physical database itself (i.e., the data actually stored on a storage device) and the users of the system, a database management system or DBMS is typically provided as a software cushion or layer. In essence, the DBMS shields the database user from knowing or even caring about the underlying hardware-level details. Typically, all requests from users for access to the data are processed by the DBMS. For example, information may be added or removed from data files, information retrieved from or updated in such files, and so forth, all without user knowledge of the underlying system implementation. In this manner, the DBMS provides users with a conceptual view of the database that is removed from the hardware level. The general construction and operation of database management systems is well known in the art. See e.g., Date, C., “An Introduction to Database Systems, Volumes I and II”, Addison Wesley, 1990; the disclosure of which is hereby incorporated by reference.
Increasingly, businesses run mission-critical systems which store information on database management systems. Each day more and more users base their business operations on mission-critical systems which store information on server-based database systems, such as Sybase® Adaptive Server® Enterprise (available from Sybase, Inc. of Dublin, Calif.). As a result, the operations of the business are dependent upon the availability of data stored in their databases. Because of the mission-critical nature of these systems, system availability is a critical need for today's businesses. Within the global business environment, business continuity has become essential with 99.999% availability a requirement for most organizations. Database applications, being an integral part of any IT infrastructure, demand a high degree of availability. It is essential that a database server is made available as quickly as possible after a planned shutdown (perhaps for maintenance activities), or an unplanned shutdown due to a hardware or software fault.
Whenever a database system crashes, an “active failover” system may take over the load (of the primary system that crashed). When this happens, there is a need for the system's databases to be recovered—that is, to bring the databases back into a consistent state.
During server re-start, a significant amount of time is spent on database recovery. With present-day systems, the typical approach employed is to recover each database serially (i.e., one at a time). For example, first the system databases may be recovered, followed by recovery of user databases one at a time. Given the length of time required to recover databases serially, there is much interest in finding an improved approach.
Recently, database vendors have attempted to improve database recovery by recovering databases concurrently. Therefore, instead of recovering one database at a time, the approach entails recovering multiple databases concurrently. The maximum degree of parallelism is equal to one less than the maximum number of database engines that a system begins with. For example, a single instance of Sybase® Adaptive Server® Enterprise (ASE) running five concurrent engines may undertake four (parallel) recoveries concurrently.
When recovering multiple databases concurrently, the performance of the system is limited by the performance of the underlying Input/Output (I/O) subsystem. Since database recovery is I/O intensive, the I/O subsystem presents a potential bottleneck to overall performance. The degree of parallelism is only available if the processing itself can be “parallelized.” Since recovery is I/O bound, the recovery process may be stalled in the face of multiple concurrent I/O requests. For example, the underlying disk subsystem may experience a degree of “thrashing.” In systems that use virtual memory (where data is moved into and out of virtual memory via “page swapping”), for example, the condition results from a hard drive being used excessively for virtual memory because the physical memory (i.e., RAM) is full. Disk thrashing considerably slows down the performance of a system because data has to be transferred back and forth from the hard drive to the physical memory. With serial recovery, thrashing tends to not be a problem as the system is just recovering one database at a time (with concomitantly lower I/O load). With concurrent database recovery, however, the system may be required to concurrently process several disparate I/O requests, for the many databases that are laid out separately on hard disk. In the presence of thrashing, the performance of concurrent database recovery may degrade to the point that it is worse than serial recovery. Accordingly, this problem poses a significant challenge to successfully implementing concurrent database recovery.
Current database systems may work in conjunction with an operating system that supports parallel I/O, so that the database system may take advantage of concurrent recovery. Expectedly, the performance of such a database system is dependent on the underlying I/O subsystem. If the underlying I/O subsystem does not provide good parallel performance, then database recovery suffers. In the state of the art today, there is no way (short of direct user configuration input) in which a database system may directly determine the level of I/O subsystem support that is really available for parallel database recovery. In other words, existing database systems today are not able to figure out what degree of parallelism should be used for concurrent database recovery. Even in instances where the DBA (database administrator/user) may manually configure the degree of parallelism, any incorrect input may result in system performance for concurrent recovery that is even worse than simple serial recovery. As the current approach to parallel database recovery does not really take into account how well the underlying I/O subsystem provides parallel performance, the current approach is suboptimal.
Given the foregoing shortcomings, there is a need for a database system that takes into account how well the underlying I/O subsystem really provides parallel performance. Ideally, the system should operate in an automated manner to determine the degree of parallelism that should be used after a system crash or failover, so that the system is essentially self-tuning and thus capable of providing self-tuned parallel database recovery. The present invention fulfills these and other needs.